1. Technical Field
The present invention relates generally to data transfers in data processing system networks and in particular to transfer of large data blocks over the Internet or similar networks. Still more particularly, the present invention relates to utilizing parallel data transmission streams to improve bandwidth for transfers of large data bundles over the Internet.
2. Description of the Related Art
The Internet has become an important conduit for transmission and distribution of data (text, coded, image, video, audio, or mixed) and software. Users connect to the backbone with broadly divergent levels of performance, ranging from 14.4 Kb/s to more than 45 Mb/s. Moreover, Transmission Control Protocol/Internet Protocol (TCP/IP) has become a widely implemented standard communication protocol, enabling broad heterogeneity between clients, servers, and the communications systems coupling them.
The differences in equipment performance, user skill, and error rate (packet loss) necessitates monitoring of user data receipt and user response under the TCP/IP system. To accomplish this, TCP/IP establishes a xe2x80x9csliding windowxe2x80x9d which limits the maximum number of sequential bytes between the byte most recently from the server to the earliest byte for which a receipt acknowledge has not yet been received from the client. A variety of algorithms automatically re-send packets and slowly restart data transmission when the sliding window limit is exceeded. Thus, if the link between the server and client is shut down in the middle of transmission of a data collection, the server will stop sending packets within-one sliding window of the last packet acknowledged by the client.
This use of a sliding window inherently limits the bandwidth B of a data transfer through the backbone to the ratio:   B  =            sliding      ⁢              xe2x80x83            ⁢      window              round      ⁢              xe2x80x83            ⁢      trip      ⁢              xe2x80x83            ⁢      time      
where the sliding window is a number of bytes. The round trip time (which may be approximated by ping time) is the time for a packet to leave the sender, pass through the routers and firewalls for the sending enterprise""s server, hop through the Internet backbone from one backbone operator to another, pass through the client""s routers, and be received by the client, plus the time required for an acknowledgment to likewise pass between the client and server in the opposite direction.
One obvious component of round trip time is propagation time. Even with fiber optic cables, if the server and client are separated by 10,000 kilometers, the light pulses require 100 msec for the round trip. Another component to round trip time is transmission delays. If there are ten hops in each direction with 5 msec lost in transmission delay for each hop, another 100 msec is added to the round trip time. With additional delays from the response time at the client and any bandwidth limitations in the client and server infrastructure, a total round trip time on the order of 200 msec may be expected in such situations.
Ping times within the United States and Canada are typically on the order of 100 msec; ping times between the United States and western Europe or Japan are typically on the order of 200 msec. Typical default sliding windows in products such as IBM""s AIX 4.3 (Berkeley configuration) are 16 KBytes. As a result, the maximum bandwidth achievable for such products is about 160 KB/sec (i.e., 1.28 Mb/sec) within the United States and about half that between the United States and western Europe or Japan. Even a single T1 connection provides greater bandwidth, and many enterprises have far more than single T1 line capacity.
One approach to improving bandwidth utilization for large data transfers is to set the sliding window to extremely high values. However, the sliding window must be reset in both the server and client systems, which is possible for special situations (e.g., particular, controlled transfers between commonly-owned or controlled systems) but not for the heterogenous Internet environment, where sliding windows within a large number of client systems cannot be automatically set to match the sliding window within the server system. In distributing software to thousands of customers, for example, a wide variety of operating systems is encountered, none of which allow outside control over the sliding window. Most users are unfamiliar with the controls enabling sliding windows to be reset to higher values. Moreover, increasing the sliding window to levels at which it no longer becomes the bandwidth-limiting factor would cause other serious problems to occur in servers and networks.
It would be desirable, therefore, to improve the bandwidth utilization in large data transfers without altering the default sliding window. It would further be advantageous for the data transfer to utilize a single physical channel for the transfer.
It is therefore one object of the present invention to improve data transfers in data processing system networks.
It is another object of the present invention to improve transfers of large data blocks over the Internet or similar networks.
It is yet another object of the present invention to utilize parallel data transmission streams to improve bandwidth for transfers of large data bundles over the Internet.
The foregoing objects are achieved as is now described. Available bandwidth utilization during transfer of large files over a TCP/IP network, ordinarily constrained by the sliding window and round trip times, is improved by segmenting the file and concurrently transmitting portions in logical distinct data streams which may all utilize a single physical medium and communications path. The file is segmented to support optimal transfer utilizing any number of concurrent data streams up to and including a maximum number, if specified. An initial number of data streams, which is one unless otherwise specified or determined, is used to begin transmission of segments, each on a different data stream. During the first seconds of the transfer, individual and aggregate transmission bandwidths are determined, together with a backbone limit computed from the sliding window and round trip times (approximated by ping time). If the individual transmission bandwidths are close to the backbone limit, additional data stream(s) and segment(s) are started. Individual and aggregate transmission bandwidths are again determined, and the process is repeated until no substantial increase in aggregate bandwidth is achieved by adding an additional data stream. Aggregate bandwidths are checked upon completion of each segment to determine if a data stream should be terminated.
The above as well as additional objectives, features, and advantages of the present invention will become apparent in the following detailed written description.